Skip to content

fix(dotnet): deduplicate failing test output (#2501)#2511

Closed
outofrange-consulting wants to merge 1 commit into
rtk-ai:developfrom
outofrange-consulting:fix/dotnet-test-output-duplication-2501
Closed

fix(dotnet): deduplicate failing test output (#2501)#2511
outofrange-consulting wants to merge 1 commit into
rtk-ai:developfrom
outofrange-consulting:fix/dotnet-test-output-duplication-2501

Conversation

@outofrange-consulting

@outofrange-consulting outofrange-consulting commented Jun 19, 2026

Copy link
Copy Markdown

Summary

Fixes #2501rtk dotnet test printed every failure twice on failing runs, inflating output by ~65% on the reported benchmark (the opposite of RTK's purpose). The regression was specific to failing runs; all-green runs were unaffected.

Root cause

In run_dotnet_with_binlog (src/cmds/dotnet/dotnet_cmd.rs), the failure branch prepended the entire raw stdout in front of the filtered output:

let output_to_print = if !command_success {
    if !stdout_trimmed.is_empty() {
        format!("{}\n\n{}", stdout_trimmed, filtered)  // ← prepends FULL raw stdout
    } ...
};

For test, that collides with the filter's own output. The raw stdout already prints every per-test failure inline (xUnit v3 / MTP), and format_test_output independently rebuilds the same failures into its Failed Tests: section — so each failure appeared once from the raw prepend and once from the structured section. build / restore source failures from a single place, so they never duplicated.

Fix

Extract the output-composition decision into a pure, unit-testable compose_dotnet_output. For test, the raw-stdout prepend is dropped only when the structured section is a complete substitute — i.e. failed_tests is non-empty and every failure carries detail. Otherwise the raw fallback is kept exactly as before, so nothing is ever lost:

Scenario failed_tests Behavior
Failing test, detail parsed (TRX/console) populated, detail present drop raw prepend → deduplicated
Failure with empty details (self-closing UnitTestResult) populated, detail empty keep raw fallback ✅
Build failure / crash / no TRX / counts-unavailable empty keep raw fallback ✅
build / restore n/a keep raw fallback ✅ (unchanged)
Success n/a filtered only (unchanged)

The .all(|t| !t.details.is_empty()) guard (not .any()) preserves the raw message for name-only failures. Behavior is byte-for-byte identical to the previous inline logic in every case except the de-duplicated one.

Tests

The bug lived in the orchestration path, which had no test coverage (existing tests only exercised format_test_output in isolation). Added 8 regression tests covering every row above, including a benchmark-shaped case asserting the failure assertion string appears exactly once (the issue's measured proof: raw=5, buggy rtk=10, fixed=5).

Verification

cargo fmt --all -- --check   # clean
cargo clippy --all-targets   # No issues found
cargo test --bins            # 2216 passed; 0 failed; 8 ignored

On a failing `dotnet test` run, run_dotnet_with_binlog prepended the
entire raw stdout in front of the filtered output. The raw stdout already
prints every per-test failure inline (xUnit v3 / MTP), and
format_test_output independently rebuilds those same failures into its
`Failed Tests:` section — so each failure was printed twice, inflating
output by ~65% on the reported benchmark (the opposite of RTK's purpose).
The more failures, the worse the linear duplication.

Extract the output-composition decision into a pure, unit-testable
`compose_dotnet_output`. For `test`, drop the raw-stdout prepend only when
the structured section is a complete substitute — i.e. failed_tests is
non-empty and every failure carries detail. When any failure lacks detail
(self-closing UnitTestResult, crash with no TRX, counts-unavailable run)
or for build/restore, the raw fallback is kept exactly as before, so no
information is ever lost. The `.all()` guard (not `.any()`) preserves the
raw message for name-only failures.

The bug lived in the orchestration path, which had no test coverage; the
existing tests only exercised format_test_output in isolation. Add eight
regression tests covering every row of the analysis, including a benchmark
-shaped case asserting each failure string appears exactly once.
@CLAassistant

CLAassistant commented Jun 19, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@outofrange-consulting

Copy link
Copy Markdown
Author

Closing as fixed in #2502

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dotnet test filter duplicates failure output on failing runs

3 participants